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ESTIMATION OF THE VARIANCE OF PARTIAL SUMS OF 
DEPENDENT PROCESSES 

HEROLD DEHLING, ROLAND FRIED, OLIMJON SH. SHARIPOV, DANIEL VOGEL, 

AND MAX WORNOWIZKI 


Abstract. We study subsampling estimators for the limit variance 

OO 

= Var(Xi) + 2 ^ Gov(Xi, A*) 

k = 2 

of partial sums of a stationary stochastic process {Xk)k>i- We establish L 2 -consistency of a 
non-overlapping block resampling method. Our results apply to processes that can be rep¬ 
resented as functionals of strongly mixing processes. Motivated by recent applications to 
rank tests, we also study estimators for the series Var(F’(Ai))-|-2 Cov(F’(Ai), F{Xk)), 

where F is the distribution function of Xi. Simulations illustrate the usefulness of the 
proposed estimators and of a mean squared error optimal rule for the choice of the block 
length. 


1. Introduction and Main Results 

Let {Xi)i>i be a stationary ergodic sequence of random variables with mean fi and finite 
variance, satisfying the central limit theorem, i.e. 

where the limit variance is given by the formula 

OO 

(1) t72 = Var(Ai) + 2^Cov(Ai,Afc). 

k=2 

For practical applications of the central limit theorem, e.g., for calculation of confidence 
intervals or testing hypothesis concerning the mean, we need to estimate the variance ci^. 

We investigate the properties of a subsampling estimator of a. Given observations 
Ai,...,A„ and a block length I = 1^, we define the overall mean and the block means 
as 

- n ^ ^ il 

-V = -EU' “d yS.(') = | E V. 

1=1 j={i-l)l+l 

respectively. By the central limit theorem, {Si{l) — lfM)/Vl converges in distribution to 
A(0,cj^) as I goes to infinity, and thus E\Si{l) — I ^\/Vl Thus, it is natural to 
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estimate a by the arithmetic mean of ^ i = \^,,, ^\jri/l'\. Replacing the unknown 

mean /i by the sample mean we obtain 


( 2 ) 


Br, 


1 

M 



IXn 


as an estimator for a. In what follows we will always write kn = [n/ln\ for the number of 
full blocks of length In in the set {1,..., n}. 

Subsampling estimators of the limit variance of the arithmetic mean of a stationary 
stochastic process have been studied by various authors under different assumptions con¬ 
cerning the dependence structure. Carlstein (1986) investigated a-mixing processes and the 
estimator 


( 3 ) 


Sp) = 


[n/l] 


[n/l] 

E 

i=l 


\Si{l)-lXn 

VI 


for cr^. Peligrad and Shao (1995) studied properly normalized means of p-th powers of 


Si{l) — lXn\/Vl as an estimator for cr^ in the case of p-mixing processes. They consider 
overlapping blocks, i.e. 




where Si{l) = 

Peligrad and Suresh ( 

1995 

) studied the same estimator, in the 

case p = 1, for associated processes. Doukhan et al. (2010 

) investigated the case of weakly 

dependent (in the sense of 

Doukhan and Louhich 

1 , 1999) processes. 


In this paper, we extend the above mentioned results to processes that can be represented 
as functionals of strongly mixing processes. We assume that (Xj)j>i can be written as 
Xi = /((yj+fc)fcez) for all i > 1, where / : —)• R is a measurable function and where 

(Ej)j>i is an a-mixing process. Recall that a process is called a-mixing (or strongly 

mixing) if 

Qffc = sup sup \P{Ari B) — P{A) P{B)\ ^ 0 
n>i 

as n —)• oo. Here denotes the cr-held generated by the random variables Yk, Tfc+i, • • •, Wn. 
The coefficients ak are called mixing coefficients. In addition, we have to require that the 
function / : R^ —)■ R defined above is continuous in a suitable sense. Define 

2+S 

tpra = E\f{{Yi^k)kez)—E{f{{Yi^k)k^'z)\Yi_rn,---,Yi+rn)\'^+X 
= E\f{{Y,+k)k&)-E{f{{Yi+k)k&^)\Yi-m,...,Yi+m)\. 


In what follows, we will have to make assumptions concerning the behavior of the coefficients 
ctm, 1pm and (pm as m —)• oo. 

Functionals of a-mixing processes cover most of the standard examples of weakly depen¬ 
dent processes known in the literature, for example ARMA- and GARCH-processes from 
time series analysis, many Markov processes and hidden Markov models, the sequence of 
digits and remainders in continued fraction expansion, and many chaotic dynamical sys¬ 
tems, such as expanding, piecewise monotone maps of the unit interval. Details can be 
found, e.g., in Borovkova et al. (2001) and in Bradley (2007). 
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Theorem 1.1. Suppose (Xfc)fc>i is a stationary process that can he expressed as a func¬ 
tional of an a-mixing process. Suppose that < oo for some 5 > 0 and that the 

approximation coefficients {'tfk)k>i satisfy 

OO 

^ ^ l-l-(5 

< oo 

k=l 

and the mixing coefficients {ak)k>i satisfy ak = 0{k~^) for some /x > (2 + S)‘^/6. Then, 
for any sequence {ln)n>i satisfying In oo and In = o(n), we have Bn ^ cr in L 2 . 


Dehling and Fried (2012) study robust non-parametric tests for structural breaks in time 


series. They consider the model Xj = + /Xj, where (fi) is a stationary stochastic process 

and (/Xi)j>i is a sequence of unknown constants. The hypothesis of interest is 


ddo • /Xi — ... — fin, 


to be tested against the alternative fii = ... = fim 7 ^ Tni+i = ■ ■ ■ = Tn- Investigating the 
asymptotic distribution of the two-sample Hodges-Lehmann test statistic 


Qm, 712 ( 0 . 5 ) = median{(Xj — Xi) : 1 < i < ni,ni 1 < j < ni 7 x 2 }. 
Dehling and Fried show that under the above null hypothesis, we have 


H\0) 


1 nin2 

Qni,772(1/2) 

y ni+n2 

1/4 


iV(0,l) 


as ni,n 2 —)• 00 . Here H{x) = P{Xi — Xi < x), where Xi is an independent copy of Xi, 
and 


(4) 


4 = Var(F(Xi)) + 2 ^ Cov(F(Xi), F(Xfc)), 


k=2 


where F is the distribution function of Xi. For statistic al ap plications, the limit variance 
4 has to be estimated. We can directly apply Theorem 1.1 to the sequence {F{Xk))k>i, 
provided that it satisfies the conditions and that F is known. The latter is generally not 
the case and thus F has to be replaced by the empirical distribution function Fn. In what 
follows, we will show that this leads to an L 2 -consistent estimator. 

Denoting the empirical distribution function by Fn{x) = 4 l{Xi<a;}) define the 

estimators 

\Ti{ln)-lnUn\ 

i=l 
[n/ln] 

(5) Dn = E 

i=l 


Dn = 


1 

[n/lr, 

1 

[n/lr, 


y/ln 

\Ti{ln) - In Un 


y/ln 


where Hn = - Un = - El=i ^n(Xj), and 


r,(0 = ^ Fix,), fill) = ^ FniX,). 

j=(i-l)l+l j=(i-l)l+l 
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Th eore m 1.2. Let {Xk)k>i be a stationary process satisfying the assumptions of Theo¬ 
rem 0 In addition, assume that an = 0{n 4>ni = 0{m and thatF{x) = P{X < x) 
is Lipschitz-continuous. Then, as n ^ oo. In oo and In = o{y/n), we have Dn —>• op in 

L2- 


2. Simulation study 


We investigate the performance of the proposed estimators and compare them to alter¬ 
natives. The estimators are applied to ARM A (1,1) processes with Gaussian innovations 
and different parameter settings, which are given in Table For each parameter combina¬ 
tion, 1000 samples are drawn, and variance, bias and mean squared error of the estimates 
are computed. The sample size is n = 500 throughout. Besides comparing several esti- 


AR 

0.5 

0.1 

-0.1 

-0.8 

0 

0 

0 

0 

0.5 

-0.5 

0 

MA 

0 

0 

0 

0 

0.8 

0.1 

-0.8 

-0.1 

0.5 

-0.5 

0 


Table 1. Types of ARMA(1,1) time series used in the simulation. AR and 
MA denote the corresponding coefficents of the ARMA(1,1) model in the 
usual notation. 


mators, another question addressed by the simulations is the applicability of Carlstein’s 
rule for choosing the block length. Carlstein ^986 ) proved it to be optimal in terms of 
mean squared error for his estimator Bn ^, cf. ^ , in case of an AR(1) process with known 
autocorrelation parameter. 

The simulation study consists of two stages: the first part is concerned with the estimation 
of (T^, cf. Q, the second part with the estimation of ap, cf. (0. 

Part 1. We compare Bn, cf. the Carlstein estimator L^\ referred to as CE in the 
following, and the kernel-based estimator of de Jong and Davidson (2000), abbreviated by 
JDE in the following. While the former two depend on the block length, the latter is affected 
by the choice of the kernel and the bandwidth 7 „. We use the Bartlett kernel, which is a 
common choice due to guaranteed positive semi-definiteness. 

To compute mean squared error and bias we compare the estimates to the true value of 
Var(;^ ~ + i Z]fc =2 Cov(Ai, Xfc)(n-|-1 —/c), not the infinite series (jl ). We 

use the fixed block lengths 1, 2, 5, 10, 30, 50 and 100, and determine the data adaptive block 
length following Carlstein as follows: An AR(1) process is fit to the sample (regardless of 
whether it was generated as such) using the R function arimaO (R Development Core Team[ 


2011), and the block length is computed from the estimated autocorrelation parameter. For 


the JDE, the bandwidths 2.8, 3.5, 4.7, 7.9 and 22.4 are used, corresponding to sixth to second 
root of n = 500. 

Part 2. The second part of the simulation, which deals with the estimation of ap, compares 


Dn, cf. ([^, to the Peligrad and Shao (1995) estimator, denoted by PSE in the following. 
Both depend on a block length, but contrary to Dn, the PSE is based on overlapping 
blocks. The data generation is exactly the same as in the first part (sample size, number 
of repetitions and ARMA parameters), but the data evaluation is different: Each sample is 
split into halves and the estimate of interest is computed as the mean of the two subsample 
estimates. This ch o ice is motivated by the intended application to the change-point test by 
Dehling and Fried (2012). The block lengths considered are 2, 5, 10, 25, 40, 50 for Dn and 


5, 10, 25, 40, 45 for the PSE. Note that, while a block length 1 is of interest in the first 
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block length block length 

Figure 1. Estimated bias (left) and mean squared error (right) of the es¬ 
timators Bn, CE and JDE as a function of the smoothing parameter; com¬ 
puted from 1000 samples of size 500 generated from an ARMA(0.5, 0.5) 
model. Gray symbols correspond to the data adaptive block length, plotted 
at the mean block length of 16.04. The x-axis is on log scale. 


simulation, since it yields a variance estimator for independent observation, it leads to data 
independent estimators in this situation and is hence neglected. 

For computing the true value of ap we make use of the fact that the population version of 
Spearman’s rho gs{X, Y) = 12E {F{X)F{Y)) — 3 equals ^ arcs in (|) if A and Y are jointly 
Gaussian with correlation p. This result can be traced back to Pearson (1907); for a newer 
reference see, e.g., Groux and Dehon ( |2010 ). Hence, letting pk denote the correlation of Xi 
and Xk, we have 


Cov{F{Xi),F{Xk)) = ^Qs = ^ arcsin 


Results. As far as the comparison of the estimators is concerned, we get similar results for all 
ARMA parameter settings, which are exemplified for the cj^-estimators at an ARMA(0.5,0.5) 
process in Figure]^ and for the (T|.-estimators at an MA(O.l) process in Figure]^ The per¬ 
formance of the estimators is generally quite similar. This is also true for Dn and the PSE, 
which may seem surprising, since the PSE, in contrast to Dn, uses overlapping blocks. In 
terms of bias. Bn and Dn slightly outperform their competitors (see left panels of Figures 
[^and[^, but they show slightly bigger variances, leading to a worse mean squared error 
(see right panels). The choice of the smoothing parameter (block length or bandwidth) has 
altogether a much larger influence than the choice of the method. We furthermore observe 
the following: The data adaptive block lengths, which correspond to the gray symbols in the 
hgures, perform quite well in terms of mean squared error (MSE) unless the MA-coefficient 
is strongly negative. Except for a few cases, ci^ is always underestimated if all autocorrela¬ 
tions of the underlying process are non-negative and overestimated if all are non-positive. 
In the white noise case, a block length of 1 is MSE optimal, but other block lengths give 
very similar results. On the other hand, even for small non-zero AR- or MA-coefficients, 
not accounting for the dependence substantially impairs the accuracy of the estimation. 
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Figure 2. Estimated bias (left) and mean squared error (right) of the esti¬ 
mators Dn and PSE as a function of the block length; computed from 1000 
samples of size 500 generated from an ARMA(0.5, 0.5) model. Gray sym¬ 
bols correspond to the data adaptive block length, plotted at the mean block 
length of 2.7. The x-axis is on log scale. 


In summary, all estimators considered show a very similar performance for good choices 
of the smoothing parameter - despite some considerable differences in their construction. 
An unfavorable choice of the block length leads to an increase in both, bias and variance, 
but with varying extents for the different estimators. Based on our results, we suggest to 
use the estimators Bn, Dn if one is willing to trade a smaller bias for a bigger variance. The 
data adaptive block length proposed by Carlstein (1986) seems to yield acceptable results 
for all block length dependent estimators even if the assumption of an AR(1) process is not 
met. We also carried out some simulations for other values of n. While the estimators’ per¬ 
formance, for fixed block length respectively bandwidth, clearly improves with the sample 
size, the ranking of the estimators is little affected. Therefore we focus on a fixed sample 
size here. 


3. Proofs 

We will first state and prove a special case of Theorem |1.1| where the process (Xj)j>i is 
a-mixing. We need this result later on in the proof of Theorem [Tt} 

Proposition 3.1. Suppose (Xfc)fc>i is a stationary, a-mixing process with 
E{\Xi\'^~^^) < oo for some 5 > 0 and mixing coefficients {ak)k>i satisfying ak = 0{k~^) 
for some ^ > (2 -|- 5)‘^/5. Then, for any sequence {ln)n>i satisfying In oo and In = o{n), 
we have Bn ^ cr in L 2 . 


Proof. We will show that E 


Bn ^ 


2 

—)■ 0 as n —>• OO. By definition of Bn, we get 




Bji (T 

< 


1 \Si{ln) — In h\ 

y/Tn 



+ Y -^\/~im{Xn — h) 
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As In = o(n), we get by the central limit theorem \/T^{Xn — //) —?■ 0 in L^. Thus, it remains 
to prove that 


1 

lE 


2=1 


TT I Si {In') In 


— a 


0 . 


By Lemma 4.3 we have 


and thus it suffices to show 


E 


\Si{ln) - InlA 




TT a 


E 


1 \Si{ln) - In ^ f\Si{ln) - InfJ^l 


E 

2=1 


y/Tn 


- E 


y/Tn 


0 . 


Applying Lemma 4.1 with t = 0 to the stationary a-mixing process {Si{l))i>i, we obtain 

2 

^ 1 ^ - L//J - KIAALI - L//J 


1 \Si{ln) - ln^^\ - E\Si{ln) - ln^i\ 

jc 2^ 


i=l 


y/ln 


1 


m 

c 


-E 


M - M) 


2=1 


< — (e| (|5l(/n) - ln^^\ - E\Sl{ln) - ln^^\) 


2+|\ 2+5/2 


< 


c 

k In 


(^E\Si{ln) - lnf^(^ 


2+A 2+'5/2 


c 

< — 
- k 


{e\Xi\ 


2+5\2+5 


where we have used Lemma 4.1 with t = 6/2 in the final step. As A; = A:„ —?• oo, we have 
thus proved Proposition |3.1[ □ 


Proof of Theorem Defining 

Us) = E{X,\Elll), Us) =X,- EiX^Ul+l), 

we decompose (Xj)j>i into a part that is a-mixing and a remaining part, which is small. 
In addition, we define the corresponding arithmetic means U — ~ 

n '221/=i Vi{s) and the block sums 

il il 

Siif,i)= E Uv,i)= E 

j=(.j-i)/+i j=(j_i)/+i 

Furthermore, let 

OO 

o-2(s) = Var(^i(s)) + 2 ^ Cov(6(s), ^j(s)). 

i=2 

For fixed s, the process Us) is a-mixing with coefficients ai- 2 s- Moreover, we have 
< oo and E{\r]i{s)U^) < oo. Finally, one can show that 

|Cov(77o(s),r?/(s))| = O > 






















H. DEHLING, R. FRIED, O. SH. SHARIPOV, D. VOGEL, AND M. WORNOWIZKI 


see Ibragimov and Linnik (1971). Observe that 

( 6 ) 




Bn-O- 

< 


^2 2.-^- 


2=1 


VI 


+ \a - a{s) 


(/,??) -IVn 

2 = 1 


Vi 


< 


^- 


2=1 


VI 


+ \a - a{s) 




2 = 1 


Vi 


We will now bound each of the terms on the right hand side separately. To the third term, 
we apply the inequality 


E ( < Var(r/i(s)) + 2^ ( ^ “ 7 ) Cov(??i(s), 77 ^( 5 )) 


VI 


i=2 


< 


I 

(2N + l)Var(,i(«)) + 2 


5/(2+<5) , ^dl+'5)/(2+<5)\ 


j=N 


valid for any 2 < N < 1. We fix N and choose sq sufficiently large such that for all s > sq 


E 


Si{l,r]) - lEr]i{s) 


VI 


<e" 


and hence by Minkowski’s inequality 


l^\S,{l,r^)-lE{m{s))\ 

k 2-^ 


2=1 


Vi 


< e. 


Since cr(s) —)■ cr, we also have |it(s) — (t\ < e for all s > sq with sq sufficiently large. 
Proposition |3 .1 1 implies that, for any hxed s, 


1 R^\ssV)-iV 

kV 2^ 


2=1 


Vi 


a{s). 


in L 2 as n —?• 00 . Hence the first ter m on the right hand side of (j^ converges to 0 in L 2 . 
Finally, stationarity and Lemma 
hnishes the proof of Theorem 1.1 


4.2 


imply Vi\r]n — .£'(^i('S))| —)• 0 in L 2 as n —)■ 00 . This 

□ 


Proof of Theorem ] 1.^ As F is Lipschitz continuous, the process {F{Xk))k>i satisfies the 
conditions of Theorem [Ml hence Dn ap in L 2 hy Theorem |1.1[ Thus it suffices to show 
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that Dn — Dn —)• 0 almost surely. We get 


\Dn - Dn\ < 


1 


[n/ln]Vla\ 2 ^ 


[n/lr, 


Y, min)-fi{ln)-ln{Un-Un) 


[n/lr, 


< 




2=1 


< 2 In sup \Fn{t) - F{t)\. 

teR 


Lemma 


4.4 


implies \Dn — Dn\ —)• 0 almost surely, provided In = o( iogi^gn )- This proves that 
Dn is an L 2 consistent estimator of the limit variance ap- D 

4. Auxiliary results 

For ease of reference we state some results from the literature on weakly dependent 
processes that we need in the course of the proofs. 


Lemma 4.1. (Yokoyama, 1980) Let (Yj)j>i be a stationary sequence of random variables 
with EXi = 

00 

< 00 and 'Y, («fc) < 00 , 


k=l 


for some 0 < 5 < 00 and 2 < t < 2 + 5. Then 


E\Yi^i - t)\^ < Cn 


2 +s\ 2 +s 


k=l 


Lemma 4.2. (Shao and Yn, 1993) Let be a sequence of random variables with Ef^n = 

0 and sup Ef,l< 00 . Assume that there is a constant C>0 such that sup E (j2i=k+i ^i) ^ < 
Cn for any n > 1. Then 


lim sup 


rjn log^ n 


= 0 a.s. 


Lemma 4.3. (Ibragimov and Linnik, 1971) Let be a stationary sequence of random 

variables with Efi = 0, < 00 and < 00 for some 6 > 0. Then 

(T^ = Ef^f + 2 k^ase > 0, 

(^1 + ...+ ^n) A^(0,1) as n —)• 00 . 


awn 


Lemma 4.4. (Philipp, 1977) Let {^n)n>i be a stationary sequence of strongly mixing ran¬ 
dom variables with an = 0{n~^), and let rjn = f{f,n,f,n+i,---), n > 1 , be a sequence of 
functionals with common distribution function F{t), satisfying 

fjm = E\r]n - E{rjn\TYn\ = Oim-Y, 
where En^"^ = a {f,n,, fn+m) • Then 

V^iFnit) - Fit)) ^ 

hm sup sup- . =- < c 

n^oo teR V21oglogn 
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almost surely, where Fn{t) = ^ Ya=i 


References 

S. Borovkova, R. M. Burton, and H. Dehling. Limit theorems for functionals of mixing 
processes with applications to R-statistics and dimension estimation. Transactions of the 
American Mathematical Society, 353(11):4261-4318, 2001. 

R. C. Bradley. Introduction to strong mixing conditions. Vol. 1-3. Heber City, UT: Kendrick 
Press, 2007. 

E. Carlstein. The use of subseries values for estimating the variance of a general statistic 
from a stationary sequence. Annals of Statistics, 14:1171-1179, 1986. 

C. Croux and C. Dehon. Influence functions of the Spearman and Kendall correlation 
measures. Statistical Methods and Applications, 19(4):497-515, 2010. 

R. M. de Jong and J. Davidson. Consistency of kernel estimators of heteroscedastic and 
autocorrelated covariance matrices. Econometrica, 68(2):407-424, 2000. 

H. Dehling and R. Fried. Asymptotic distribution of two-sample empirical ?7-quantiles with 
applications to robust tests for shifts in location. Journal of Multivariate Analysis, 105 
(1):124-140, 2012. doi: 10.1016/j.jmva.2011.08.014. 

P. Doukhan and S. Louhichi. A new weak dependence condition and applications to moment 
inequalities. Stochastic Processes and their Applications, 84:313-342, 1999. 

P. Doukhan, J. Jakubowicz, and J. R. Leon. Variance estimation with applications. In 
1. Berkes, R. Bradley, H. Dehling, M. Peligrad, and R. Tichy, editors. Dependence in 
Probability, Analysis and Number Theory., pages 203-231. Heber City, UT: Kendrick 
Press, 2010. 

I. A. Ibragimov and Y. V. Linnik. Independent and stationary sequences of random variables. 
Groningen: Wolters-Noordhoff, 1971. 

K. Pearson. Mathematical contributions to the theory of evolution. XVI. On further methods 
of determining correlation. Drapers’ company research memoirs: Biometric series IV. 
London: Dulau &: Co., 1907. 

M. Peligrad and Q.-M. Shao. Estimation of the variance of partial sums for p-mixing random 
variables. Journal of Multivariate Analysis, 52(1):140-157, 1995. doi: 10.1006/jmva.l995. 
1008. 

M. Peligrad and R. Suresh. Estimation of variance of partial sums of an associated sequence 
of random variables. Stochastic Processes and their Applications, 56(2):307-319, 1995. 
doi: 10.1016/0304-4149(94)00065-2. 

W. Philipp. A functional law of the iterated logarithm for empirical distribution functions 
of weakly dependent random variables. Annals of Probability, 5:319-350, 1977. doi: 
10.1214/aop/1176995795. 

R Development Core Team. R: A Language and Environment for Statistical Comput¬ 
ing. R Foundation for Statistical Computing, Vienna, Austria, 2011. URL http: 
//www.R-project.org, ISBN 3-900051-07-0. 

Q. -M. Shao and H. Yu. Bootstrapping the sample means for stationary mixing sequences. 
Stochastic Processes and their Applications, 48:175-190, 1993. 

R. Yokoyama. Moment bounds for stationary mixing sequences. Zeitschrift fiir Wahrschein- 
lichkeitstheorie und verwandte Gebiete, 52:45-57, 1980. 



ESTIMATION OF THE VARIANCE OF PARTIAL SUMS OF DEPENDENT PROCESSES 


11 


Fakultat fur Mathematik, Ruhr-Universitat Bochum, 44780 Bochum, Germany 

E-mail address: herold.dehling@rub.de 

Fakultat Statistik, Technische Universitat Dortmund, 44221 Dortmund, Germany 

E-mail address: fried@statistik.tu-dortmund.de 

Department of Probability Theory and Mathematical Statistics, Institute of Mathematics 
AND Information Technologies, Uzbek Academy of Sciences, Tashkent, Uzbekistan 
E-mail address: osharipov@yahoo.com 

Fakultat fur Mathematik, Ruhr-Universitat Bochum, 44780 Bochum, Germany 

E-mail address: vogeldts@rub.de 

Fakultat Statistik, Technische Universitat Dortmund, 44221 Dortmund, Germany 

E-mail address: wornowiz@statistik.tu-dortmund.de 



